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Preface 


Many  methods  of  obtaining  confidence  limits  for  system 
reliability  exist  in  both  practice  and  literature.  The 
understanding  and  use  of  these  techniques  are  often  quite 
difficult.  Because  of  this,  I  embarked  on  an  effort  to 
devise  a  technique  that  was  theoretically  sound,  easy  to 
understand,  and  simple  to  use.  This  report  discusses  that 
technique  and  its  accuracy. 

Sincere  gratitude  is  extended  to  Dr.  Albert  H.  Moore 
for  his  guidance,  expertise,  and  motivation  toward  comple¬ 
tion  of  this  study.  Thanks  also  goes  to  Mr.  Joseph  J.  Meli, 
ASD/ENADC ,  who  was  my  sponsor. 


Roy  E .  Rice 
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Abstract 

Several  methods  for  estimating  lower  confidence  limits 
were  examined.  A  proposed  technique  was  developed  and  used 
to  obtain  limits  for  selected  systems.  These  limits  were 
compared  to  those  obtained  by  other  methods.  Then  the  pro¬ 
posed  method  was  tested  to  verify  its  accuracy.  Results 
indicate  that  the  proposed  method  is  simple  to  understand, 
easy  to  implement,  and  accurate. 
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INCORPORATION  OF  ASYMPTOTIC  NORMALITY  PROPERTIES  OF 
THE  BINOMIAL  DISTRIBUTION  INTO  A  MONTE  CARLO 

TECHNIQUE  FOR  ESTIMATING  LOWER  CONFIDENCE  LIMITS 
ON  SYSTEM  RELIABILITY 

I.  Introduction 

In  the  late  1920's  and  early  1930's,  prominent  statis¬ 
ticians  begem  to  realize  that  a  single  point  estimate  for 
the  probability  of  occurrence  of  an  event  was  not  suffi¬ 
cient.  Initial  studies  by  Fischer,  Neyman,  Clopper,  Pear¬ 
son,  and  many  others  show  the  necessity  and  practicality  of 
placing  probabilistic  bounds  on  these  estimates.  The 
application  of  these  intervals  injects  confidence  into  the 
accuracy  of  these  point  estimates.  Through  these  pioneer¬ 
ing  efforts  we  arrived  at  the  concept  of  confidence  inter¬ 
vals. 

This  analysis  intends  to  introduce  some  of  the  more 
important  historical  works  that  address  the  calculation  of 
confidence  limits.  It  contains  considerably  detailed  dis¬ 
cussions  of  the  evolution  of  various  techniques  for  deriv¬ 
ing  confidence  limits  with  major  emphasis  on  intervals  for 
binomial  distributed  parameters.  There  are  discussions  of 
techniques  involving  Bayesian  Limits  using  Mellin  Trans¬ 
forms  (Ref  22:611),  Approximately  Optimum  Limits  (Ref  18: 
335-347),  Monte  Carlo  simulation  (Ref  19:459),  and  others. 
There  is  then  a  discussion  of  the  proposed  method  of  incor¬ 
porating  the  asymptotic  normality  assumption  into  a  Monte 
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Carlo  technique.  Finally,  a  verification  technique  is 
included  that  shows  how  accurate  the  confidence  limits  are. 

Component  test  data  is  obtained  where  the  number  of 
failures  is  assumed  to  be  distributed  binomially.  Then  the 
components  are  arranged  in  a  logically  complex  system  and 
an  estimate  of  the  reliability  and  confidence  limit  is 
made.  These  estimates  from  the  proposed  method  are  then 
compared  with  estimates  from  the  other  methods.  A  final 
effort  is  made  to  verify  the  accuracy  of  the  limits  by  sim¬ 
ulating  the  testing  of  the  components  and  determining  the 
number  of  times  the  limits  cover  the  true  reliability. 

Of  particular  interest  is  the  treatment  of  test  data 
in  which  no  failures  occur.  Many  of  the  methods  are  not 
valid  when  the  data  shows  zero  failures.  Others  simply 
disregard  the  samples  with  zero  failures.  Still  others 
manipulate  the  data  to  circumvent  the  problem.  Almost  all 
of  this  results  in  a  biasing  effect  on  the  estimates.  This 
study  explores  another  way  of  handling  samples  that  exhibit 
no  failures. 

Coupled  with  the  improvement  of  these  estimating  tech¬ 
niques  is  the  advancing  computer  technology.  High  speed 
computers  make  it  increasingly  easier  and  more  efficient 
to  make  detailed  computations  and  simulations.  Thus,  it  is 
becoming  more  practical  to  apply  these  methods  to  systems 
exhibiting  pass/fail  characteristics  from  the  simple 
(switches,  pins,  relays)  to  the  very  complex  (missile 
flights,  docking  systems,  targeting). 

2 


1 


1 


In  the  following  study  there  are  several  basic  assump¬ 
tions.  It  is  assumed  that  the  reader  is  familiar  with  the 
concept  of  confidence  limits /  the  binomial  distribution, 
and  has  a  basic  understanding  of  statistics  and  reliability 
pertaining  to  both  components  and  systems. 
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In  one  of  the  first  studies  in  this  area,  Springer  and 
Thompson  (Ref  22:611)  examined  the  posterior  density  func¬ 
tions  of  (i=l , 2 , . . . ,N) ,  the  binomial  parameters  of  N 
independent  subsystems,  when  the  prior  distribution  is  uni¬ 
form  or  Beta.  The  Bayes  posterior  density  function  of  R^ 
in  the  uniform  prior  case  is 


f .  (R.  ) 
1  l 


(ni+l)  ! 
mi! (ni~mi) ! 


m.  n.-m. 

R,  1 (1-R. )  1  1 


(0  <  R±  £  1,  i=l , 2 , . . . ,N) 


where  nu  =  number  of  successes  in  ni  trials. 

Epstein  gives  the  Mellin  integral  transform  of  f^R.^) 
as 


M{fi(Ri)  j  s } 


(n^+1) i  (s+m. ) 

in~!  (s+n.+l)  '  Re(s)  >  mi  * 


Huntington  shows  that  the  Mellin  convolution  of  f^CR^)  and 
f2(R2)  is  exactly  the  probability  density  function  of  the 
product  R1R2  and  that  the  repeated  convolution  of  the 
probability  density  functions  is  equivalent  in  the  trans¬ 
form  to  repeated  multiplication  of  the  Mellin  transforms. 
From  this.  Springer  and  Thompson  derive  a  closed  form  of 
the  posterior  distribution  function  for  N  independent  sub¬ 
systems  in  series,  H(R).  At  this  point,  they  emphasize 
that,  for  large  N,  computation  of  H(R)  is  tedious  for  hand 
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computations  and  extensive  using  computers.  Thus,  they 
conclude  that  some  asymptotic  method  might  warrant  investi¬ 
gation.  For  more  detailed  investigation  of  their  method, 
see  Reference  22. 

Later,  in  a  study  by  Mann  and  Grubbs  (Ref  18:335-347), 
confidence  bounds  for  system  reliability  are  derived  based 
on  component  test  data  when  the  components  are  assumed  to 
be  (1)  exponentially  distributed  with  censoring  or  trunca¬ 
tion  of  tests  for  a  fixed  number  of  failures,  (2)  exponen¬ 
tially  distributed  with  truncation  of  tests  at  fixed  times, 
and  (3)  binomially  distributed  (pass-fail)  with  fixed  but 
different  sample  sizes  and  random  numbers  of  failures  for 
subsystem  tests.  With  regard  to  the  binomial  distribution, 
Mann  arrives  at  approximate  expressions  for  m  and  v  ,  the 

5  S 

mean  and  variance  of  -In  Rg  which  corresponds  to  these 
optimum  nonrandomized  confidence  bounds  for  series  systems. 
These  are 

•s<^>  ,  p 
"s  n°  (1-.5P) 


and 


.5(l-±)n» 

3L  S 


n 


where  n°  =  (1-.5P2)  (1-.5P) 


n^j  is  the  smallest  of  the  n^  ,n2  , .  •  •  ,n^. 
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p 


k  „ 

1  -  it  R.  is  the  max- likelihood  estimate  of  the 
j=l  3 

probability  of  failure  of  the  series  system. 


a  =  n^j£(l/nj)  which  is  a  restricted  sum. 


The  rij's  for  all  zero-failure  subsystems  are  omitted 

in  calculating  the  value  of  a  =  n^Kd/n^)  except  for  a 

single  zero-failure  subsystem  with  its  sample  size  equal  to 

n^j.  Mann  and  Grubbs  point  out  that  this  subsystem  is 

also  ignored  if  at  least  one  of  the  other  subsystems  which 

exhibit  some  failures  has  the  smallest  sample  size  n^*, 

or  if  the  lower  confidence  bound  obtained  by  not  ignoring 

it  is  larger  than  that  obtained  by  ignoring  it. 

"The  exclusion  of  some  subsystems  in  calculating 
series-system  confidence  bounds  also  is  necessi¬ 
tated  when  all  sample  sizes  are  large  and  most 
of  the  failures  pertain  to  a  single  subsystem. 

In  this  case  the  bound  obtained  based  on  all  sub¬ 
system  failures  can  be  larger  than  that  based  on 
the  single  subsystem  exhibiting  nearly  all  the 
failures."  (Ref  18:337) 

In  their  analysis  they  compare  their  bounds  with  those 
based  on  Buehler's  exact  methods,  those  based  on  maximum 
likelihood  normal  approximations,  the  modified  maximum 
likelihood  procedure  of  Easterling,  and  the  likelihood 
ratio  method  of  Madansky.  These  last  three  procedures  can¬ 
not  be  applied  effectively  when  one  or  more  subsystems 
exhibit  zero-failures.  This  comparison,  along  with  the 
estimates  from  the  proposed  method,  is  shown  in  Tables  I, 
II,  III,  and  IV. 
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Another  technique  that  is  of  extreme  importance  is 
that  developed  by  Gatliffe  (Ref  7) .  Mathematically,  Gat- 
liffe  begins  by  stating  that,  for  systems  with  k  components 
in  series. 


R_  = 


7T  P.  ,  P.  is  the  reliability  of  the  i 
i=l 

component 


,  th 


Qi 


1  -  P. 
1 


Let  s  -  -In  R  =  -  Z  ln(l  -  Q. ) 

s  i-1  1 

k  Q2  q3 

s  =  r  (q.  +  -r  +  -r  +  •••  ) 
i-l  1  2  J 

k  Q?  k 

*  Z  (Q,  +  -±)  -  1  T,  ,  T, 


i»l 


i-l 


Qi 

(Qi  +  -£>  . 


The  unbiased  estimator,  + 


(2Ni  "  3) 

Ai  =  (2Ni  -  2) 

N. 

B  ~  i 
Bi  nL  -  l  ' 

the  number  of  failures 
resulting  from  mis¬ 
sion  tests  conducted 
on  ifck  component . 
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He  establishes  another  unbiased  estimator 


*  ~ 

s  =  IT. 

i-1  1 

A  k 

with  Var(s)  =  Z  (T./N.)  . 

i=l  1  1 

He  intuitively  assumes  the  probability  distribution  of 
gamma  with  parameters  r  and  0 , 


1  xr  e-x/t>  ,  x  >  0 


rr-er 


f-(x,r,9)  = 


with  mean  =  r9 ,  variance  =  r9  . 


,  x  <  0 


E(s)  =  re  -  Z  T . 

i=l  1 


Var(s)  =  r9‘ 


Z  (T./N.) 
i-1  1  1 


r  = 


(ETi)4 

i  — 

L  N. 

l 


0  = 


STi/Ni 

ZT- 


r  = 


(ET.r 

ET./N. 

i  i 
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<  W 


r 


t/N«».  A  2s  o 

Since  s  is  distributed  as  gamma  (r,9),  =^-  will  be  X (2r) * 

From 

1-0,  =  p[^  >  and 

l-a  =  p|*9s^f-  1  and  since  9  =  5iii_  and 

L  X(2r,l-oOJ 

A 

E(s)  =  -ln(R  ),  he  has  obtained  a  100(l-a)%  lower  confidence 

S 

limit,  R  (a)  ,  for  R 
s  s 


R£(a) 


exp 


~  -2rs  1 
y2 

.  (2r,l-a) J 


A  100 (1-a) %  lower  confidence  limit  estimate,  Ri (a) ,  for  R 

^ s 

would  be 


R|(a) 


It  is  apparent  that  a  problem  arises  in  the  event  of 
no  failures  in  any  components.  This  yields  an  s  value  of 
zero,  thus,  a  system  reliability  confidence  limit  estimate 
of  one  (1) . 

Gatliffe  outlines  his  procedure  to  deal  with  such  an 
event  as  follows : 

(1)  Find  the  series  component  with  the  smallest  actual 
or  equivalent  number  of  trails,  N  or  N'. 

(2)  Change  the  number  of  component  or  subassembly 
failures  to  F'  by  the  following  scheme: 
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a 


I 


F1 

.2  .37 

.1  .25 

.15  .16 

(See  derivation  in  Appendix  A. ) 

(3)  Recompute  the  lower  confidence  limit  estimate 
using  the  basic  log-gamma  procedure  with  the  revised  fail¬ 
ure  data.  This  estimate  is  approximately  equivalent  to 

,  \  1/N 
(a) 

Gatliffe's  scheme  to  arrive  at  equivalent  failures  was 
used  in  the  proposed  method.  It  is  an  amazingly  simple, 
yet  effective  tool;  and,  for  this  reason,  it  is  recommended 
that  the  reader  refer  to  Appendix  A. 

The  techniques  discussed  in  this  chapter  were  not 
used  directly  in  the  proposed  method.  Yet,  they  give  dif¬ 
ferent  approaches  that  are  of  historical  and  analytical 
importance.  They  are  primarily  doorways  that  open  up  into 
a  vast  and  intriguing  area  of  study. 
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III.  Proposed  Method 

The  essence  of  this  study  is  to  examine  a  series-system 
with  components  that  experience  failures  distributed  binom- 
ially  and  derive  a  simple,  yet  accurate,  method  for  esti¬ 
mating  confidence  limits  for  system  reliability.  The 
method  of  obtaining  lower  confidence  limits  that  is  proposed 
in  this  study  draws  upon  two  important  concepts?  the  asymp¬ 
totic  normality  properties  of  the  binomial  distribution  and 
Monte  Carlo  simulation.  Both  warrant  discussion. 

It  has  been  shown  that,  when  a  pass-fail  component  is 
tested  a  sufficient  number  of  times  (20  or  more) ,  its  fail¬ 
ures  are  distributed  normally  with  a  mean  of  p  and  a  vari- 

/V  A 

ance  of  ESI.  in  this  treatment,  p  is  defined  as  the  proba¬ 
bility  of  success,  q  is  1-p,  and  n  is  the  number  of  trials. 
So  for  any  component  i, 

F . 

1  -  —  ,  F.  =  the  number  of  failures  of  ith 
ni  1 

component  in  n^  trials, 

A  A 

q.  =  1  -  p.  . 

Note  that  if  there  are  no  failures,  p^  =  1  and  the  variance 
equals  zero.  The  existence  of  components  with  a  reliabil¬ 
ity  absolutely  equal  to  1  is,  to  say  the  least,  a  rarity. 

To  eliminate  this  problem,  an  equivalent  number  of  failures, 
F^',  is  assigned  according  to  Gatliffe's  method  (see 
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Appendix  A) .  Once  these  initial  estimates  have  been 
obtained,  they  can  be  used  in  a  Monte  Carlo  simulation. 

This  Monte  Carlo  method  begins  by  assigning  failures 
or  equivalent  failures  to  each  component  in  a  series  sys¬ 
tem.  From  this  a  p^  is  obtained  for  each  component.  Now, 
using  the  asymptotic  normality  property,  a  random  variable 
from  a  normal  distribution,  with  mean  =  0  and  variance  *  1, 

/V  A 

piqi 

is  drawn.  If  —  is  the  asymptotic  variance,  then  the 
square  root  of  this  value  is  the  standard  deviation.  Since 
a  N (0 , 1)  random  variable  is 


Z 


9 


one  can  take  this  random  variable,  multiply  it  by  the 
asymptotic  standard  deviation,  and  add  it  to  the  mean,  pi. 
What  is  obtained  is  a  second  estimate  of  p.^, 


A 


A  -A 

N<pi' 

i 


If  this  is  done  for  each  component,  one  can  obtain  an  esti¬ 
mate  of  the  system  reliability  for  k  components  in  series. 


A 


k  ~ 

A 

*  Pi 
1*1 


A  Monte  Carlo  simulation  (Ref  9,10,15,19,20,21)  is 
used  to  generate  a  number  of  these  system  reliability  esti¬ 
mates.  These  estimates  are  then  ranked  in  ascending  order. 
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The  ordered  reliabilities  partition  the  interval  [0,1]  into 
n+1  equally  probable  intervals.  For  simplicity,  let  n+1 
equal  1000 .  From  the  ordered  simulation  reliabilities  one 
cam  obtain  any  desired  lower  confidence  limit  by  extracting 
the  100 (1-a)  percentile  value.  For  instance,  with  999 
ordered  reliability  estimates,  the  90  percent  lower  confi¬ 
dence  limit  would  be  the  99th  ordered  estimate. 

The  outline  of  this  procedure  is  as  follows: 

(1)  For  each  component  in  turn  assign  the  number  of 
failures,  F^,  or  equivalent  failures,  F|, 

(2)  Calculate  the  first  estimate 


A 


F. 

i  "  H7  or  1 


F! 

l 


i 


A 


asymptotic  variance  = 


A  A 


*1*1 


t 


(3)  Draw  a  random  variable  from  N(0,1)  for  each  com¬ 
ponent  , 

x  ~  P- 

(4)  Obtain  a  second  estimate,  p^  ^  NCp^,  *  -)  by 

ni 

taking  the  N(0,1)  random  variable,  multiply  by  the  asymp¬ 
totic  standard  deviation,  and  add  it  to  p^  for  each  compo¬ 
nent  , 

(5)  With  the  components  in  a  logically  complex  system, 

A  k  ~ 

arrive  at  a  system  reliability  estimate,  R  =  it  p.  , 

s  i=l  1 
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(6)  Repeat  steps  (3)  through  (5)  999  times, 

(7)  Rank  order  these  999  estimates  of  R  , 

s 

(8)  Extract  the  100  (1-ot)  percentile  value  to  obtain  a 
100(l-a)%  lower  confidence  limit. 

This  procedure  was  applied  to  systems  of  2  and  3  com¬ 
ponents  in  series.  The  computer  program  used  (see  Appendix 
B)  shows  that,  with  minor  adjustments,  larger  systems  can 
be  examined.  In  these  2  and  3  component  systems,  various 
numbers  of  failures  and  tests  were  analyzed.  These  partic¬ 
ular  values  were  chosen  so  as  to  give  a  basis  of  comparison 
(see  Tables  I,  II,  III,  and  IV).  The  proposed  method  pro¬ 
duces  values  that  compare  extremely  well  with  those  used 
in  the  Mann  and  Grubbs  study  (Ref  18:345). 

Several  points  in  the  tables  deserve  note.  First,  the 
values  obtained  in  this  method  most  closely  compare  with 
those  from  the  maximum  likelihood  method.  Secondly,  in 
every  case,  they  are  slightly  higher  than  the  "exact"  val¬ 
ues  which  tends  to  indicate  upward  biasness  in  the  esti¬ 
mates.  This  latter  aspect  is  what  led  to  further  investi¬ 
gation  as  to  the  accuracy  of  the  estimates.  The  next  chap¬ 
ter  examines  this  area. 
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95%  Limits,  Two  Components 


TABLE  III 

90%  Limits,  Three  Components 


95%  Limits,  Three  Components 


IV.  Verification  of  Accurae 


To  merely  derive  an  estimating  technique  and  its  sub¬ 
sequent  estimates  is  not  sufficient.  Analysis  must  be  con¬ 
ducted  to  verify  the  accuracy  of  the  method.  As  discussed 
in  Chapter  I  of  this  study,  the  concept  of  confidence 
intervals  and  limits  evolved  from  the  need  to  know  the 
accuracy  of  point  estimates.  But  it  is  obvious  upon  exam¬ 
ining  Tables  I,  II,  III,  and  IV  that,  depending  on  which 
method  is  used,  values  of  these  confidence  limits  vary  con¬ 
siderably.  This  leads  to  an  effort  to  verify  the  accuracy 
of  the  proposed  method  of  obtaining  lower  confidence  limits. 

Basic  to  the  theory  of  confidence  limits  is  the  fact 
that,  if  a  system  of  known  reliability  is  tested  a  number 
of  times,  the  estimates  of  a  100(l-a)%  confidence  interval 
should  include  the  true  system  reliability  100(l-a)%  of  the 
time.  From  this  fact  came  the  motivation  for  the  subsequent 
analysis . 

Given  a  system  with  known  reliability  that  is  made  up 
of  components  with  established  probabilities  of  success, 
one  can  apply  the  procedures  described  in  Chapter  III  to 
estimate  desired  lower  confidence  limits  and  compare  these 
limits  with  the  true  system  reliabilities.  Then,  by  gen¬ 
erating  many  of  these  limits,  it  can  be  determined  how 
often  the  true  reliability  is  in  the  interval  [LCL,1] .  For 
example,  if  1000  80%  lower  confidence  limits  are  generated, 
it  is  expected,  theoretically,  that  800  of  the  limits  are 
lower  than  the  true  reliability. 


Monte  Carlo  simulation,  again,  is  the  tool.  Given  k 
components  with  known  probabilities  of  success,  p^,  con¬ 
nected  in  series,  a  true  system  reliability  is  calculated, 

k 


Next,  one  must  specify  the  number  of  tests  that  each  com¬ 
ponent  is  to  be  subjected  to,  NTV.  To  simulate  the  testing 
of  each  component,  simply  draw  a  uniform  random  variable  in 
the  interval  [0,1]  for  each  test.  Now  compare  each  of 
these  NT^  random  variables  with  p^.  If  the  random  variable 
is  greater  than  p^,  a  failure  is  recorded.  By  doing  this 
for  each  component,  one  obtains  the  first  estimate  of  p^ 


A 


/ 


A 


asymptotic  variance  = 


p .  q. 
NT. 


As  done  in  the  previous  chapter,  if  no  failures  are  exper¬ 
ienced,  Gatliffe's  method  is  used  to  assign  equivalent 
failures.  This  is  done  for  each  component.  The  proposed 
method  then  produces  a  second  estimate  of  p^  by  drawing  a 
random  variable  from  a  N(0,1),  multiplying  by  the  asymp- 

A 

totic  standard  deviation,  and  adding  this  to  p^.  These 
estimates,  p. ,  are  then  combined  to  produce  an  estimate  of  R  , 

X  s 
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Rs  =  *  Ph 

s  i=l  1 


Again,  999  of  these  estimates,  R_,  are  generated  and  ranked 

s 

in  ascending  order.  Then  the  100 (1-a)  percentile  value  is 
extracted  to  yield  a  100 (1-a) %  lower  confidence  limit  esti¬ 
mate  ,  R  (a)  . 
s . 

: 

Now,  to  test  the  accuracy  of  this  method,  many  more 
repetitions  of  this  testing  and  limit  estimating  are  done. 
This  yields  a  number  of  lower  confidence  limits  that  can  be 
compared  to  the  true  system  reliability.  As  stated  before, 
one  would  expect  100 (1-a) %  of  these  100  (1-a) %  lower  confi¬ 
dence  limits  to  include  the  true  system  reliability. 

The  outline  of  this  procedure  is  as  follows: 

(1)  Establish  the  value  of  p^  for  each  component  and 
the  number  of  tests  to  subject  each  component  to,  NT^, 

(2)  For  each  component  in  turn,  draw  NT^  random  vari¬ 
ables  from  Uniform  [0,1], 

(3)  For  each  component,  compare  these  NT^  random  var¬ 
iables  to  p^;  for  each  random  variable  that  is  greater  than 
p^,  record  a  failure, 

(4)  Calculate  the  first  estimate 


=  1  - 


NTi 


or  1  - 


IL 

NT.  ' 

l 


«  1  - 


Pi  , 
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(5)  Draw  a  random  variable  from  N(0,1)  for  each  com¬ 
ponent, 

A 

(6)  Obtain  a  second  estimate,  ,  by  taking  the  N(0,1) 
random  variable,  multiply  by  the  asymptotic  standard  devi- 
ation,  and  add  it  to  p^  for  each  component, 

(7)  With  the  components  in  a  logically  complex  system, 

k  £ 

calculate  a  system  reliability  estimate,  R  =  ir  p.  , 

s  i=1  1 

(8)  Repeat  steps  (5)  through  (7)  999  times, 

(9)  Rank  order  these  999  estimates  of  R  , 

s 

(10)  Extract  the  100  (1-ot)  percentile  value  to  obtain  a 
100(l-a)%  lower  confidence  limit, 

(11)  Repeat  steps  (2)  through  (10)  1000  times  to  obtain 
1000  of  these  100(l-a)%  lower  confidence  limits, 

(12)  Determine  how  many  of  these  lower  confidence  limits 

k 

are  covering  the  true  system  reliability,  R  =  it  p.  . 

s  i=l  1 

The  above  procedure  was  applied  to  systems  containing 
2  and  3  components  in  series.  The  computer  program  used 
(see  Appendix  C)  shows  that,  with  minor  adjustments,  larger 
systems  can  be  examined.  In  these  2  and  3  component  sys¬ 
tems,  known  probabilities  of  success,  p^,  of  0.6,  0.8,  0.9, 
and  0.95  were  examined  to  find  80%,  90%,  and  95%  lower 
confidence  limits.  Table  V  and  Table  VI  show  the  number 
of  lower  confidence  limits  that  contained  the  true  system 
reliability  for  each  of  the  systems  with  various  values  of 
p^,  a,  and  number  of  tests. 
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TABLE  V 


Component  System 


Close  examination  of  Table  V  and  Table  VI  points  out 


that  accuracy  declines  as  gets  larger  and  as  a  gets 
smaller.  These  results  verify,  as  Mann  and  Grubbs  men¬ 
tioned  (Ref  18:343),  that  an  upward  biasness  might  exist. 

By  studying  the  computer  program  in  Appendix  C,  the 
reader  should  note  several  points.  At  higher  values  of  pi , 
some  tests  will  exhibit  no  failures.  Hence,  it  is  possible 

/v 

that  p.  can  be  greater  than  unity.  To  eliminate  this  event, 

a  -  Mi 

p^  is  chosen  to  be  the  minimum  [N(pi#  -^T  — )  ,1] .  Also,  the 
reader  should  note  that,  on  output,  he  is  provided  with  an 
average  lower  confidence  limit.  In  the  generation  of  the 
1000  lower  confidence  limits ,  they  are  averaged  to  produce 
the  most  accurate  estimate. 

To  illustrate  the  utility  and  simplicity  of  this  tech¬ 
nique,  consider  the  mission  profile  of  an  air-to-ground 
missile.  Assume  that  it  has  eight  independent  mission 
stages  with  associated  probabilities  of  success: 

(1)  Release  from  aircraft,  p-^  =  0.95 

(2)  Air  start,  P2  =  0.95 

(3)  Deploy  flight  control  surfaces,  p3  =  0.9 

(4)  Arrive  at  battle  area,  p^  =  0.95 

(5)  Terrain  following/avoidance,  p^  =  0.85 

(6)  Penetrate  defenses,  Pg  =  0.75 

(7)  Hit  target,  p7  =  0.95 

(8)  Detonate,  pg  =  0.95 

These  stages  are  analogous  to  a  system  with  eight  compon¬ 
ents  in  series.  By  multiplying  all  of  the  p^'s,  the  true 
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system  reliability  (probability  of  killing  the  target)  is 
determined  to  be  0.42059.  Exercising  the  proposed  method 
with  100  tests  per  component  with  the  above  p^'s,  a  90% 
lower  confidence  limit  for  system  reliability  is  0.3665. 

In  this  particular  case,  91.3%  of  the  1000  limits  cover  the 
true  reliability.  Therefore,  we  would  say  that  it  is  a 
very  accurate  estimate. 

Another  example  of  the  usefulness  and  simplicity  of 
this  technique  is  shown  by  estimating  the  lower  confidence 
limits  for  the  two  simple  parallel-series  systems  shown  in 
Figures  1  and  2.  In  analyzing  the  three  component  system 
shown  in  Figure  1,  the  probability  of  success  for  each  com¬ 
ponent,  p^,  was  chosen  to  be  0.9;  and  each  component  was 
tested  100  times.  This  design  yields  a  system  reliability 
of  0.891.  The  estimating  technique  provides  an  80%  lower 
confidence  limit  of  0.86842  with  the  lower  limits  covering 
the  system  reliability  in  75%  of  the  1000  Monte  Carlo  repe¬ 
titions.  This  would  indicate  that  the  LCL  estimate  is 
slightly  high. 

The  five  component  system  in  Figure  2  was  assigned  the 
following  p^'s: 

(1)  Pl  =  0.95 

(2)  p2  =  0.75 

(3)  p3  =  p4  =  0.90 

(4)  p5  =  0.80 

Calculations  yield  a  system  reliability  of  0.9006  and  a  90% 
lower  confidence  limit  of  0.8671.  These  limits  cover  the 
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Figure  1 


1 


Figure  2. 
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.  Three  Component  System 


Five  Component  System 
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system  reliability  in  87.8%  of  the  1000  cases  which  indi¬ 
cates  again  that  the  LCL  estimate  is  slightly  optimistic. 
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V.  Conclusions  and  Recommendations 

Stated  very  simply,  it  has  been  shown  that  this  method 
of  estimating  lower  confidence  limits  is  accurate,  practical, 
and  simple  to  use.  One  need  only  supply  the  component  reli¬ 
abilities,  the  number  of  component  tests,  and  the  desired 
level  of  confidence;  and  he  obtains,  not  only  an  estimated 
lower  confidence  limit  of  the  system  reliability,  but  also 
an  indication  of  how  accurate  this  estimate  is. 

It  is  worth  re-emphasizing  that,  where  most  of  the 
other  techniques  are  not  valid  in  the  case  of  zero-failures, 
this  method  accurately  and  easily  accommodates  such  a  situ¬ 
ation.  This  method  is  also  not  restricted  to  series  sys¬ 
tems;  it  can  easily  handle  parallel  configurations.  A 
logical  next  step  would  be  to  incorporate  such  a  method 
into  a  system  where  other  components  exhibit  different 
failure  distributions  as  in  the  work  done  by  Moore,  Harter, 
and  Snead  (Ref  20) . 

This  next  generation  of  systems  to  be  analyzed  obvi¬ 
ously  provokes  one  to  imagine  all  the  possible  applications. 
These  methods  could  be  applied  to  an  entire  spectrum  of 
systems  from  simple  hardware  items  to  strategic  mission 
planning. 
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Appendix  A 

Case  Where  s=0 

In  Gatliffe's  method  (Ref  7),  he  must  deal  with  the 
case  of  s=0.  In  his  study,  the  number  of  mission  successes, 
W,  can  be  related  to  mission  reliability,  Rg ,  by 

^  N  i  n— i 

I  (.)  RJ  (1-R  )  J  =  Prob  (#  successes  i  W)  , 
j=W  3  s  s 

where  N  =  number  of  mission  trials.  The  100(l-a)%  Lower 
Confidence  Limit  estimate  for  system  reliability  would  be 
the  solution  for  R*  in 

E  (!?) (R*) j (l-R*)N-j  =  a 

j=W  3 

which,  when  W=N,  reduces  to 


(R*)N  =  a  or  R*  =  (a)1//N  . 


Now, 


R*(a)  =  exp 


- 


-2rs 


(2r,l-a)  J 


=  (a) 


1/N 


where 


A 

S 


max 


1, 


(IT^)  “ 


which  reduces  to  s  =  T.  and  r  =  max(l,  , 

one  component  contributes  non-zero  failures. 


since  only 
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Now  if  r  =  1.0,  then  T^N^  £  1  implies  that 


T .  <  — 

Ni 


Ti 


A.Q.  +  B.  >„  07 
11  l/2  wl 


2N.-3 

Ai  =  "2N1-2  '  Bi 


N .  „  F. 

-- L ■■  Q  =  _i 

N.-l  '  Ui  N. 


B.  -  A. 

Thus,  (-—)  FT  +  (^)F.  -  T.  =*  0  or 
2N.  1  Ni  1  x 


N.  2N.-3 

( - i - __)F*  +  (_ - -_)F,  -  ~  <  0 

2(N.-1)(N?)  1  2(Ni  1) (Ni}  1  Ni 


Ff  +  (2Ni-3)Fi  -  2(Ni-l)  £  0 


(Fi~l)  (F±  +  2(Ni-l))  £  0  . 


The  only  positive  values  for  F^  that  can  satisfy  this  equa 
tion  are  between  zero  (0)  and  one  (1).  Also,  for  any  size 
,  2r  =  2.  Hence, 


exp 


[A  A  • 

?=?*- 

x(2r,l-a)J 


=  (a) 


1/N 


2  _  x(2r,l-a)  ...  _ 
s  -  2N  (lna) 


A . F*  B. (F* ) 2 

i  l  +  i  i 


N 


2N 


which  implies  (F*) 2  +  (2N-3)F*  +  (N-l)  lna (X^2£ , 1_a) )  =  0 
Solving  for  F*  as  a  function  of  N  and  a,  Gatliffe  obtains 
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N 

a*.  2 

a= .  1 

a= .  05 

5 

.3888 

.2670 

.1713 

10 

.3721 

.2531 

.1610 

20 

3652 

.2475 

.1570 

100 

.  3603 

.2435 

.1542 

1000 

.3593 

.2427 

.1536 
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Appendix  B 


Computer  Program  to  Determine  Limits 

The  computer  program  contained  herein  determines  the 
100(l-a)%  lower  confidence  limits  on  system  reliability. 

It  is  written  in  FORTRAN  IV  and  utilizes  AFIT  and  IMSL  sub¬ 
routines.  At  the  beginning,  each  variable  is  defined. 
Throughout  the  program,  major  sections  are  introduced  with 
comments  that  explain  what  is  being  done  so  as  to  enhance 
the  flow  of  the  program. 

Input ;  The  first  data  card  is  in  free  field  form  containing: 
Cl)  the  number  of  components  in  the  system, 

(2)  the  confidence  level  (a  =  0.05  means  a  95%  level) , 

(3)  DSEED  which  is  the  double  precision  random  number 
seed  used  in  the  subroutines, 

(4)  the  number  of  simulations  to  be  performed  (999). 

Following  this  card  is  a  card  for  each  component,  in 
format,  that  establishes  the  number  of  tests  and  number  of 
failures  for  that  component. 

Output :  On  output  is  displayed: 

(1)  the  number  of  components, 

(2)  the  number  of  tests  and  failures  for  each  component, 

(3)  the  confidence  level, 

(4)  the  100(l-a)%  lower  confidence  limit. 
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-f  •  )  ,  ■'‘■Y's  J(i,  '  > ,  rr  H/.T2  f  =  )  ,  rI  "  1ST  Nl  T  E'3T  (•.  C  )  , 

i  ‘  <  '•),"(  r  > 


p  nitem-  <«  j  tested* 

:  f:'E‘T  =  •*  or  cent  i  failed  it.  tlst 

o  p;~£-  s  <jT  of  a  pJr.  pomojnemt  I 

c  fi>-a~2  =  icon i  of  p  po-5  rjiFu.;-K‘T  T 

C  f^ViJs  :'.-:U4a!Ll^v  IP  “YSTFM  C.l  *ONTE  CA‘LO  STY  J 

c  r-0Cw=  #  ijp  CC  1nDNE’lT3  ?N  THE 

0  F t_ »- K*  =  i.  :•  d-K^M  ■'  CrNFInENCE  LHI* 

c  Fx=  pijivalE'. r  p<  iL'r-r 

D  '’ThjTr  FH3T  ESTI^'E  fF  0  FOl’i  D0M3DNENT  I  T°=l 

r  r : ■: r ^ ^  fe  off  \i:x iance 

O  r3  -  Fr  =  3E  E"'  F)fc  nni-iri  <.v.  GENERATION 
c  r  =  uuicjf  y/Ecro;  op  unr mal  r.v.p 
C  os:>-r  «  CP  IGITE  C-*’l.o  SIMULmTIOYs* 

C  YAlc»-A=  THAT  /ALt'E  TOR  £SPC10I.«G  TO  l  HIT 

PtifHiT  lY,r-‘  .  7) 

Pu.-  Vi  T  (Pe.") 

Rr^ni  .  i:0M,  A_PH.‘  ,  D3"Er,NSTM 

IF  <  if  P  (juIN^UT  5  STOP  “END  CF  FROO  =  AM” 

VO.  IF  COMMON  EM  If.  TESTED  AND  NO.  OF  FAILURES 
DO  1C  I=1,NCCM 

READ  2,  (NITESTC),CI-PS7(I)  ) 

T  CONTINUE 

c  :is*.  *  , “INPUT  corr^...ro.  CF  COMP*  “,NCOK,"  NO.  CF  TESTS  AND  FAIL” 
P- •  I r-,7  *■  ,  *•  •• 

P“  If.T  2,  (  NiTP ST  < I)  »  eIT EST  (I)  ,I=l,HCOM 

ASSIGNING  I'DUIVAI  ENT  FAILURES  HIT H  GATuIFFE'S  METHOD 

I P  (C  LFWi  .£".  J.2>  rr_#7tu 

Zr  (ALFHA  .EO.j.l)  FT  =.2h5 
I r  (f  LFHA  .EC.  ).  T‘  )  aT  r.lpf 
DO  2t  I*1,NCCM 

:e  (FIT  EST  (I)  .  ED.  *•)  FI'EST(I)=FI 
1  CONTINUE 

rSLCULAT.NG  P,  D,  A JD  VARIANCE 
CL  ?C  I=l,‘lCG1 

FiHA’  «:)=!-( FIT -f  TO  )/<NITEST(T)  ) 
ni  R“AT  (i)  *1-PH*T(I> 

SIGIMATd)sPHAT(I)  ‘DI^AT(I)/NITFST(I) 

I  COITJNUE 

GENERATING  normal  PAMDOM  VAr<iA3LFS  AND  SECOND  ESTIMATE  OF  P 

CO  ft  J*1,NSIH 

call  GGN'*l  (DS  EED  » NCDM  >  r ) 

CD  »  C  I  =  l,NCC-t 

PI  HAT  2 (x ) =R(i)*3nFT (S  Tr I  HAT (I) ) +3IH A  T (I) 

CO  NTT  f  UE 

CALCCLATxNG  CYST3M  PLIABILITY  (SERIES) 

DC  <-c.  I  =  2,NCOM 


Appendix  C 

Computer  Program  to  Verify  Accuracy 

The  computer  program  contained  herein  determines  the 
100(l-ot)%  lower  confidence  limit  on  system  reliability  and 
determines  how  accurate  this  limit  is.  It  is  written  in 
FORTRAN  IV  and  utilizes  AFIT  and  IMSL  subroutines.  At  the 
beginning,  each  variable  is  defined.  Throughout  the  pro¬ 
gram,  major  sections  are  introduced  with  comments  that 
explain  what  is  being  done  so  as  to  enhance  the  flow  of  the 
program. 

Input :  The  first  data  card  is  in  free  field  form  containing 

(1)  the  number  of  components  in  the  system, 

(2)  the  confidence  level  (a  =  0.05  means  a  95%  level), 

(3)  DSEED  which  is  the  double  precision  random  number 
seed  used  in  the  subroutines , 

(4)  the  number  of  simulations  to  be  performed  (999) , 

(5)  the  number  of  lower  limits  to  be  generated  (1000) . 

Following  this  card  is  a  card  for  each  component,  in 
format,  that  establishes  the  true  probability  of  success 
and  number  of  tests  for  that  component. 

Output :  On  output  is  displayed: 

(1)  the  true  probability  of  success  and  number  of  tests 
for  each  component, 

(2)  the  true  system  reliability, 

(3)  the  confidence  level  and  number  of  limits, 
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\ 

1 

I 

(4)  the  averaged  100(l-a)%  lower  confidence  limit,  j 

(5)  the  accuracy  of  the  limit  (the  percent  of  the 
total  limits  that  covered  the  true  system  relia¬ 
bility)  . 
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FRGGFA*  RINREL1  (IJO'JT, OUTPUT) 

01  HE)  SION  PIHATtl?)  ,RSVSJ(iJUC>  ,? (lJ^O)  »  FI  TEST  ( 1 L  )  ,NITEST(13), 
1CIHAT  (10)  .SIGIHA7 (11)  ,  =IHAT2  (  3 , 11  1 ))  ,  TRUEP  ( 10 )  , RT EST  ( 1  Oflu) 

DOUBLE  PRECISION  OSEEO 

*♦*****♦♦**♦♦**♦**♦♦»»*••**»--  **♦**♦♦**«»  *  +  *  +  +  *»»»■++  ++**■*  *  +  + 

NlTESTs  #  OF  COMPONENT  I  TESTED? 

FITEST=  *  OF  COMPONENT  I  FAILED  IN  TEST 

PI  HAT  =  FIRST  ESTIMATE  CF  P  FOR  COMPONENT  I 

PI  HAT  2-  SECOND  ESTIMATF  OF  P  FOR  COMPONENT  I 

PS YSJ=  RELIABILITY  OF  SYSTEM  ON  10NTE  CARLO  SIM  J 

NCCM=  n  OF  COMPONENTS  TN  THE  SYSTEM 

ALPHA  =  luO(l-AL=HA) X  CONFIDENCE  LIMIT 

FI=  E CLI VALENT  FAILURES 

01  HAT  =  FIRST  ESTIMATE  OF  0  FOR  COMPONENT  I  tO=l-P 
SIGIHAT*  ESTIMATE  OF  VARIANCE 
OSEEO=  SEED  FOR  NORMAL  R.V.  GENERATION 
R=  OUTPUT  VECTOR  OF  NOFMAL  P.V.S 
NSIM=  #  OF  MONTE  CARLO  SIMULATIONS 
KALPHA=  THAT  VALUE  CORF ESPCNOING  TO  LIMIT 
NO VEF  =  NO  OF  LCL  OVER  THE  TRUE  RELIABILITY 
NL I M= NUMBER  OF  LOWER  CONFIDENCE  .IMITS 
TRSYS=THE  TRUE  SYSTEM  FELlABILITr 
RT  EST  =ARR AY  CONTAINING  TEST  RESULTS  OF  THE  PI'S 
REL=  RELIABILITY  OF  THE  SYSTEM  FOR  THAT  SIMULATION 
CUMLMs  ADOS  ALL  THE  LCL 'S 

AVGLl M=  AVERAGES  THESE  LIMITS  OVER  THE  TOTAL  NO.  GENERTTED 

** **♦#***#  ♦#*♦*♦**♦* ****  ****  ♦**#»****•  -»*+**»-**** ♦*♦♦+**#  ♦**♦♦**♦*♦ 


FORMAT  (2X,F6.4,1X,I3) 

FORMAT  (F8.6) 

REACH  *  NCCM, ALPHA, OSEEr,NSIM,NLIM 

IF  (EOF (5 LINP’JT ).NE.3.D  STOP  "END  OF  PROGRAM** 


TRUE  PRC B  OF  SUCCESS  ANO  NO.  OF  TESTS  FOR  EACH  COMPONENT 

TRSYS=1.Q 
DO  10  Isl.NCOM 

READ  2,  (TRUEP(I) ,NITE?T(I>) 

TRSYS  =  TRSYS*TRUEP (I) 

PRINT* , "THE  TRUE  VALUE  OF  P  FOR  COMPONENT  "*I,"  =  ”, TRUEP (I) 
PRINT* ,”NUMBER  OF  TESTS  ON  COMPONENT  ”,I,”  =  ”,NITEST(I> 

0  CONTINUE 

PRINT*,”  ” 

PRINT*, “THE  TRUE  SYSTEM  RELIABILITY  IS  ",TRSYS 


NOW  TESTING  COMPONENTS  WITH  UNIFORM  FUNCTION 

NOVEPsG 

CUMLM*C.O 

00  100  LIMITS=1, NLIM 
CO  20  Msl.NCOM 
NsNlTEST(M) 

CALL  GGUBS(OSEEO,N,RTEST) 


C  *■•**♦***  **•■♦#♦***♦♦**»*■**»#*  *♦*»♦*♦♦**  *4*  *»■***•♦*»■*  ♦*♦»*« 

C  PTEST  CONTAINS  TEST  VALUEC  FOR  COMP3NEJTS 
C 

FITEST  (M) =3.  0 
DO  3C  K*  1  ,N 

IF(BTEST(K)  .GT.TRUSO(M)  )  FITEST  CH>  =FITEST (M>  +1 
30  CONTI  ME 
C 

C  ASSIGNING  EQUIVALENT  FAILURES  WITH  3 ATLI FFE *S  METHOD 
C 

IF  (ALPHA. EO. 0.2)  FT*. 360 
IF  (ALPHA, EO. 0.1)  R?s , 245 
IF  (ALPHA. SO. 3. 05)  FI=.155 
IF  (FITEST (i) .EQ.O.O)  FITEST (H) a^I 
C 

C  CALCULATING  ESTIMATES  OF  F,  Q,  AND  VARIANCE 
C 

PI  HAT (M)  =  1.0-(FITEST(M)) /(NITEST(M) ) 

01 HAT (M)=1.0-PIHAT(M) 

SI GIHAT ( H ) *PIHAT ( M) *QIHAT (M) /NITEST (M) 

20  CONTINUE 
C 

C  GENERATING  NORMAL  RANDOM  VARIABLES  AND  SECOND  ESTIMATE  OF  P 
C  CALCULATING  SYSTEM  PELIA97 LITY  (SERIES) 

C 

DO  50  1*1  *  NCOM 

CALL  GGNML(DSEED,NSIM,F) 

00  AC  J*1 ,  NSIM 

PIHAT2(I.  J)sPIHAT(I)  -(P(  J)*SQRT(SIGIHAT(I)  )) 

IF (FI HAT2  (I , J) .GT.1.0)  PIHAT 2 (I , J ) *1. 0 
AO  CONTINUE 

53  CONTINUE 

DO  6(1  J=1,NSIM 
REL*1.0 

DO  70  1*1 *NCOM 
REL=PEL*PIHAT2(I,  J> 

70  CONTINUE 

RSYSJ  (J)=REL 
60  CONTINUE 

C 

Q ***♦♦♦♦***  *♦♦*♦**♦*♦*»  *»***>  »***»«***»  Mt****IM*f**»4«#***t*M**»«*4** 

C  ARRANGING  SYSTEM  RELIABILITY  IN  ASCENDING  ORDER 
C 

CALL  SORT (NSIM, RSYSJ) 

KALPHA*ALPHA»NSIM 
T*l. 0-ALPHA 

IF(RSYSJ(KALPHA) .GT.TRSYS)  NOVER*NC VER+1 
CUMLM*CUMLM*RSYSJ(KALPHA) 

100  CONTINUE 

AVGLI.MsCUMLM/NLlM 

PERCENT*NLIM-NOVER 

PERCNT  =  (PERCENT /NLIM)*10Q,0 

PRINTS , -NUMBER  OF  LOWER  LIMITS  Ar  ALPHA*  ",T,”  IS  "»NLIM 
PRINT*, "AVERAGE  LOWER  LIMIT  OF  Sf STEM  RELIABILITY  IS  -, AVGLIM 


PRINT*  ,**CERCE4T  OP  LOWER  LIMITS  ICVERING  TME  TRUE  REL  IS 
PRINT*,**  ** 

PRINT*  ,**  ** 

GO  TO  5 


1 

,PERCNT 


VITA 
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